A Visual Sensing Approach to Estimate Material Hauling Cycles in Heavy Construction and Surface Mining Jobsites

Dump trucks are the main mean for hauling earth material within and outside of the construction and open-pit mining jobsites. The real-time monitoring of the material transportation trips is beneficial for proactive equipment management, volume estimation, and productivity measurement. The traditional manual observation is tedious, non-real-time, and expensive; therefore remote sensing technologies, namely GPS and RFID, have been employed for this reason. Ordinary construction videos are another costeffective alternative to monitor dirt loading and hauling operations in real-time. A vision-based system has been developed to recognize and count hauling trips. This system has three modules including object detection, tracking, and action recognition. This system processes videos taken from access roads in which it recognizes dump trucks and then tracks them throughout the viewfinder. The action recognition module is designed to reject false alarms and also terminate tracking. This way, the system is able to count the number of hauling trips and also record the time between each trip. Several state of the art computer vision algorithms were employed to develop this system. Some of these algorithms were modified using spatiotemporal information to minimize the inherent errors of the vision-based methods. The experimental results showed considerable performance of the system for practical application in the construction and mining sectors.